The present invention relates generally to mosaicing algorithm, and relates more particularly to a method for mosaicing frames from a video sequence acquired from a fibered confocal scanning device.
Fibered confocal microscopy (FCM) is based on the principle of confocal microscopy which is the ability to reject light from out-of-focus planes and provide a clear in-focus image of a thin section within the sample. This optical sectioning property is what makes the confocal microscope ideal for imaging thick biological samples. The adaptation of a fibered confocal microscope for in vivo and in situ imaging can be viewed as replacing the microscope objective by a flexible microprobe of adequate length and diameter in order to be able to perform in situ imaging. For such purpose, a flexible fiber bundle is used as the link between the scanning device and the miniaturized microscope objective (see FIG. 1). The Cellvizio©, developed by Mauna Kea Technologies (MKT), is a complete fibered confocal microscope with a lateral and axial resolution comparable with a standard confocal microscope. It is based on the combination of:                a flexible optical microprobe consisting in a bundle of tens of thousands of fiber optics, whose overall dimensions are compatible with the accessory channel of a standard endoscope;        a proximal laser scanning unit, which assembles the functions of light illumination, signal detection, and XY robust and rapid scanning;        a control and acquisition software providing real-time image processing.        
The laser scanning unit, performs a scanning of the proximal surface of the flexible optical microprobe with the laser source by using two mirrors. Horizontal line scanning is performed using a 4 kHz oscillating mirror while a galvanometric mirror performs frame scanning at 12 Hz. A custom synchronization hardware controls the mirrors and digitizes, synchronously with the scanning, the signal coming back from the tissue using a mono-pixel photodetector. When organized according to the scanning, the output of the FCM can be viewed as a raw image of the surface of the flexible image bundle. Scanning amplitude and signal sampling frequency have been adjusted to perform a spatial over-sampling of the image bundle. This is clearly visible on the raw image in FIG. 2 where one can see the individual fibers composing the bundle. A typical fiber bundle is composed of 30,000 fiber optics, with a fiber inter-core distance dic=3.3 μm, and a fiber core diameter of 1.9 μm. Fiber arrangement is locally quasi hexagonal, but does not show any particular order at larger scales.
Each fiber of the bundle provides one and only one sampling point on the tissue. Associated with these sampling points comes a signal that depends on the imaged tissue and on the single fiber characteristics. The role of the image processing is first to build a mapping between the FCM raw image and the fibers composing the image bundle. Once the mapping between the raw data and each individual fiber is obtained, characteristics of each fiber are measured and the effective signal coming back from the tissue is estimated.
The input of the mosaicing algorithm will therefore be composed of non-uniformly sampled frames where each sampling point corresponds to a center of a fiber in the flexible fiber bundle. A collection of typical frames acquired in vivo is shown in FIG. 3.
Fibered confocal microscopy (FCM) is a promising tool for in vivo and in situ optical biopsy. This imaging modality unveils in real-time the cellular structure of the observed tissue. However, as interesting as dynamic sequences may be during the time of the medical procedure or biological experiment, there is a need for the expert to get an efficient and complete representation of the entire imaged region. The goal of the present invention is to enhance the possibilities offered by FCM. Image sequence mosaicing techniques can be used to provide this efficient and complete representation and widen the field of view (FOV). Several possible applications are targeted. First of all, the rendering of wide-field micro-architectural information on a single image will help experts to interpret the acquired data. This representation will also make quantitative and statistical analysis possible on a wide field of view. Moreover, mosaicing for microscopic images is a mean of filling the gap between microscopic and macroscopic scales. It allows multi-modality and multi-scale information fusion for the positioning of the optical microprobe. FCM is a direct contact imaging technique. In order to image and explore a region of interest, the optical microprobe is glided along the soft tissue. The displacement of the optical microprobe across the tissue can be described by a rigid motion. Since FCM is a laser scanning device, an input frame does not represent a single point in time. In contrast, each sampling point corresponds to a different instant. This induces motion artifacts when the optical microprobe moves with respect to the imaged tissue. Furthermore, the interaction of the contact optical microprobe with the soft tissue creates additional small non-rigid deformations. Due to these non-linear deformations, motion artifacts and irregular sampling of the input frames, classical video mosaicing techniques need to be adapted.
However the invention has a broader scope because it can apply to non-fibered confocal scanning microscope or to any scanning device providing a video sequence.
Another aim of the present invention is to improve the gain in both noise level and resolution of input frames whatever the acquisition system used.